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ROBUSTNESS PROPERTIES OF DIMENSIONALITY REDUCTION WITH 

GAUSSIAN RANDOM MATRICES 

BIN HAN AND ZHIQIANG XU 


Abstract. In this paper we study the robustness properties of dimensionality reduction with Gaussian ran¬ 
dom matrices having arbitrarily erased rows. We first study the robustness property against erasure for the 
almost norm preservation property of Gaussian random matrices by obtaining the optimal estimate of the 
erasure ratio for a small given norm distortion rate. As a consequence, we establish the robustness property 
of Johnson-Lindenstrauss lemma and the robustness property of restricted isometry property with corruption 
for Gaussian random matrices. Secondly, we obtain a sharp estimate for the optimal lower and upper bounds 
of norm distortion rates of Gaussian random matrices under a given erasure ratio. This allows us to establish 
the strong restricted isometry property with the almost optimal RIP constants, which plays a central role in 
the study of phaseless compressed sensing. 


1. Introduction and Motivations 


In this paper we are interested in investigating various robustness properties of dimensionality reduction 
with Gaussian random matrices having arbitrarily erased rows. Then we shall use the results to study the 
robustness properties of the Johnson-Lindenstrauss lemma and restricted isometry property. 


Throughout the paper, A = (aj,A;)i<j<m,i<fc<n £ M mxn will be a Gaussian random matrix such that 
each entry is an independent identically distributed (i.i.d.) random variable under the standard Gauss¬ 
ian/normal distribution jV(0,1) with zero mean and unit standard deviation. For T C {1, ... ,m}, we shall 
adopt the notation At £ M^ xn to denote the \T\ x n sub-matrix of A by keeping the rows of A with the 
row indices from T, where \T\ is the cardinality of the set T. Let xq £ M n be a fixed vector with ||a;o|| = 1, 
where ||xo|| is the Euclidean norm of the vector xq. For e > 0 and 0 < j3 < 1, we define 


(1.1) n e>( g := tl ej p(A, xq) := | |7pj-|| Apxoll 2 — 1 < e for all T C {1 ,..., m} satisfying \T C \ < /3m j , 


where T c := {1,... ,m}\T. For every fixed e > 0, it follows from the definition in (11.111 that P(fl e ^) is a 
decreasing function in terms of /?, where the probability is taken over the Gaussian random matrix A. 


It is well known in the literature by standard tail-bounds for the chi-squared distribution (e.g., see [H 
Lemma 4.1]) that 


( 1 . 2 ) 


P{i||Ax 0 || 2 > 1 + e} < e -F74- £ 3/ 6)m 
P{i||Ax 0 || 2 < 1 - e} < e —F 2 /4-, 3 /6)m 


Vm£N, 0 < e < 1. 


Consequently, with high probability, a normalized Gaussian random matrix -7= A has the following almost 
norm preservation property: 

(1.3) P(a,o) = IP{|^||Axo|| 2 -l| <e} > l-2e- (e2/4 - £3/6)m , Vm€N,0<e<l. 

The inequality in (j 1.3 II also implies the Johnson-Lindenstrauss lemma (see EE!)- For N points p \,... ,pw £ 
M n and for 0 < e < 1, the Johnson-Lindenstrauss lemma says that for m = O(^M^), there exists a projection 
matrix A € W nxn such that the following almost norm preservation property holds: 

(1-4) (1 - e)\\pj -Pkf < \\Apj-Ap k \\ 2 < (1 + e)\\pj -p k \\ 2 , VI <j,k<N. 
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To establish the above almost norm preservation property in (11.41) . the projection matrix A is often taken to 
be a random matrix so that the almost norm preservation property in (11.31) holds with high probability. The 
Johnson-Lindenstrauss lemma is a fundamental technique to reduce the dimensionality of the data and has 
many applications in information theory, machine learning and algorithms (c.f. fB] and references therein). 

In the compressed sensing literature, the restricted isometry property (RIP) matrix is also related to 
«■ For x € M n and s € N, we say that x is s-sparse if x has no more than s nonzero entries. Under a 
measurement matrix A 6 K mxn , we have y := (yi,... , y m ) T := Ax with m measurements y\,... ,y m - To 
successfully recover the unknown sparse signal x from the measurement vector y, it is important for the 
matrix A to satisfy the restricted isometry property (RIP) with a small positive RIP constant 0 < e s < 1: 

(1.5) (1 — e s )||u|| 2 < ||Au|| 2 < (1 + e s )||u|| 2 , for all s-sparse vectors v € R n . 

The above restricted isometry property with a small positive RIP constant e s is often established by con¬ 
sidering A to be a random matrix such as a normalized Gaussian random matrix so that the almost norm 
preservation property in () 1.3 D holds with high probability for e = e s (see [3][8]). 

When /3 > 0, we suppose that at most (3m rows of the Gaussian random matrix A are arbitrarily erased. 
It is of interest in both theory and application to study how large is the erasure ratio (3 so that a normalized 
Gaussian random matrix with any arbitrarily erased (3m rows still has the almost norm preservation 

property with high probability. Particularly, we are interested in the following two problems: 

Problem 1 : Give 0 < e < 1 and 0 < a < 1, what is the maximum (3 so that 

P(ft £) 0) > 1 - 3e -“P 2 /4- e 3 / 6 )m > for all m € N _ 

Problem 2 : Given 0 < (3 < 1 and a > 0, what is the minimum e so that 

P(J) £j ^) > 1 — 2exp(— am), for all m € N. 

Let us briefly explain our motivation for considering P(D ei( g) with (3 > 0 in the setting of Johnson- 
Lindenstrauss lemma and of compressed sensing. In Johnson-Lindenstrauss lemma, note that each projected 
vector Apj has m entries. The projected vectors are often transmitted through network by m parallel 
channels, that is, each entry of Apj is transmitted through an independent channel in a parallel manner. If 
some channels are out of work, we can only receive the corrupted projected vectors AtPj instead of Apj for 
j = 1 ,... ,N, where T C. {1,, m} is an unknown subset with \T C \ < (3m for some given corruption/erasure 
ratio 0 < (3 < 1. Consequently, it is important to first establish the almost norm preservation property with 
high probability in (11.31) for fl e p with (3 > 0. The compressed sensing with corruption considers the problem 
that a certain portion of the obtained measurements yi,...,y m are missing or corrupted by sparse noises (e.g., 
see mmm and references therein). In other words, one can only obtain the measurements Atx for some 
unknown subset T C {1,... ,m} such that \T C \ < (3m for some given corruption/erasure ratio 0 < (3 < 1. 
Therefore, it is important that the matrices At have the restricted isometry property with a small positive 
RIP constant e s for all subsets T C {1,... , m} with \T C \ < (3m. Particularly, in compressive phase retrieval, 
to recover sparse signals from the magnitude of the linear measurement, one introduces the concept of strong 
RIP which requires the matrices At satisfy the RIP property for all subsets T C {1,..., m} with \T C \ < (3m 
(c.f. m)- For example, in m, the authors considered the case (3 = 1/2. To achieve this robustness property, 
it is natural to first establish the almost norm preservation property with high probability by replacing 
and e in (11.31) with Iie s ,/3 and e s , respectively for /3 > 0. 

We first consider Problem 1. To study how large is the erasure ratio /3 so that a normalized Gaussian 
random matrix A with arbitrarily erased (3m rows still has the almost norm preservation property with high 
probability, we introduce a quantity /3™^ x to characterize the largest possible such erasure ratio (3 with a 
given fixed high probability rate a > 0. For e > 0 and a > 0, we define 

(1.6) PZT ■= sup{0 < & < 1 : P(n e>/3 ) > 1 - 3e -«P 2 /4-6 3 /6)m for all m € N }^ 

If the above set in the right-hand side of (11.61) is empty, then we simply define (3™* := 0. Due to (11.31) and 
C Q e 0 for all 0 < (3 < 1, it makes sense for us to only consider 0 < a < 1. The multiplicative constant 
3 before /i-A/Q)m - n (jjjjjj no ^ essential and can be replaced by any absolute constant greater than 

2. For simplicity of presentation, we stick to the constant 3 in (11.61) . 
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For e > 0 and 0 < /3 < 1, a closely related notion to in (11.11) is 

(1.7) (l e j := Q ej/3 (A,x 0 ) := {|^||A T a;o || 2 - l| < e for all T C {1,... ,m} satisfying \T C \ < /3m} . 

That is, we used the uniform normalization factor A f or in (11.71) instead of the factor for 3 in 
m- Similar to (11.61) . for e > 0 and a > 0, we define 

( 1 . 8 ) f max := sup {0 < 0 < 1 : P(H £)/3 ) > 1 - 3 e -«P 2 /4-e 3 /6)m for all m € 


For the case e —> 0 + (that is, e is small for the almost norm preservation property in (II. 3D ), we have the 
following result. 


Theorem 1.1. Let A be an m x n random matrix with independent identically distributed entries obeying 
A7(0,1). For every 0 < a < 1, 


(1.9) 

and 

( 1 . 10 ) 


1 - \/OL 


32 


1 - \[OL 


)f <A " 


< 


2 + 2g s 

c 2 g e g 


In - 


1 ’ 


0 < e < min(^f e g ,4e 2 ) 


32 


In 


ifT<4T< (£2 Iff. 0 < e <min(^,c 2 ln2,^), 


4c^ J In ■ 


where c g and e g are absolute positive constants given in \3.15\) and \2.10\) . respectively. 


Theorem o shows that = 0( ln ^, ) has the optimal order when e is small enough. Hence, Theo¬ 

rem 11.11 presents a solution to Problem 1 up to a multiplicative constant provided that e is small enough. 
As a direct consequence of Theorem 11.11 (more precisely, Theorem 13. 5p , by the standard argument in the 
literature for proving the Johnson-Lindenstrauss lemma using random matrices, we have the following robust 
version of the Johnson-Lindenstrauss lemma. 

Corollary 1.2. Let 0 < a < 1 and 0 < e < 1 f° . Let N,m,n £ N such that m > f/ 6 )^ ' ^ e ^ ^ ^ e 

an m x n random matrix with i.i.d. entries obeying A7(0,1). For any given N points pi, ■ ■ ■ ,pn E M n ; with 
probability at least 1 — | N(N — 1 ) e - Q ( e2 / 4 - e3 / 6 ) m > 0 , 

(1.11) (1-e)\\pj - p k \\ 2 < ±\\A T Pj - A T p k \\ 2 < (1 +e)\\pj - p k \\ 2 , Vl<j,k<N and T E T £)Q ,, 

where T e , a is defined to be 

(1-12) r e , a :=Jrc{l,.,m} : \T C \ < m • 

Another consequence of Theorem 13.51 is the following result on the robust restricted isometry property. 

Corollary 1.3. Let 0 < a < 1 and 0 < e < 1 f° . Let s,m,n £ N satisfy s In < a(e 2 /16 — e 3 /24 )m — 
ln3. Let A be an m x n random matrix with i.i.d. entries obeying A7(0,1). With probability at least 

2 _ ^^24eny e -a(e 2 /16-e 3 /24)m ^ g 

(1.13) (1 — e)||u || 2 < i||Ar'i ;|| 2 < (1 + e)||u|| 2 , V s-sparse v E M n and T E T e / 2j0l , 

where T e / 2 ,a defined in \1.12\) . 

We now turn to Problem 2, which is also related to erasure robust frames (see m)- For a given 0 < /3 < 1, 
we would like to determine the minimum e so that jpj-||Arxo || 2 £ [1 — e, 1 + e] with high probability for all 
T C {1,... ,d} satisfying \T C \ < /3m. For this purpose, we consider the most general case instead of the 
particular subsets in (11.11) . Recall that xq E W 1 with ||xo|| = 1. For 0 < /3 < 1 and 0 < 6 < uj < oo, we 
define 

(1.14) := ^[e,u>]A A ^ x o) '■= {tttII^^oII 2 € [6,u\ VTC{l,...,m} satisfying \T C \ < /3m}. 

Obviously, in (11.11) simply becomes 1 +e ] !( g. For 0 < (3 < 1 and a > 0, we define 

(1.15) 0™ ax (a) := sup{0 < 9 < 00 : P(fi[e),oo],/ 3 ) > 1 — exp (—am) for all m E N}, 

(1.16) cv^ m (a) := inf{0 < u < 00 : > 1 — exp (—am) for all m E N}, 
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and 


(1.17) 9f ax := sup {9f ax {a) : a > 0} and uf* := inf{w^ in (a) : a > 0}. 

A simple observation from the above definitions is that 0“ ax (a) < 0™ ax < cj™ in < w“ m (a) and 

(1.18) p (^[ 0 -- (a) ^min H]i/3 ) > l-2exp(-am), V m € N. 

If 0 < 0™ ax (a) < l j™ m (a) < 2, then Problem 2 is solved with e = max(l — 0™ ax (a), w™ m (a) — 1) > 0. Similar 
to (J1.14I) . we define 

(1.19) &[0,oj],P := {mll^^oll 2 G VT C {1,... ,m} satisfying \T C \ < /3m} 

and we can define 0 “ ax ,ch“ m similar to @™ ax ,cn™ m , respectively by replacing 11 with Cl. 

We now briefly explain why we are interested in Cltg^vp- An m x n matrix A is said to have the strong 
restricted isometry property of sparse order s € N and level [9, uj, (3] with O<0<cn<2,O</3<lif 

(1.20) 0|H | 2 < i||Aru || 2 < o;||u|| 2 , V s-sparse »€K" and T C { 1 ,..., m} with \T C \ < [3m. 


The strong restricted isometry property plays a critical role in the study of phaseless compressed sensing 
in EM. In P3], the authors investigated the case where (3 = 1/2 with showing that the Gaussian matrix 
has the strong RIP of order s and level [0 q,wo, 1/2] with high probability provided m = 0(s log en). Here 
9q and uq are absolute constants. The original motivation for this work is to extend the result in m 
to the arbitrary (3 € [0,1)- To show that there indeed exists a measurement matrix A having the strong 
restricted property of sparse order s and level [9,u,f3\ in (11.201) . the matrix A is often constructed by an 
mxn Gaussian random matrix with i.i.d. entries obeying A7(0,1) and one would like to have T^w^i/g) > 0 
for 0 < 9 < u < 2 with the largest possible 9 and the smallest possible w. That is, if we can prove the 
inequalities 0 < 0™ ax < w™ in < 2, for any 9 , u satisfying 0 < 9 < 0™ ax < < u < 2, as we shall prove 

in Corollary 11.61 (11.201) holds with high probability. Thus, the desired inequalities 0 < 0™ ax < cl;™ m < 2 
guarantees the strong restricted isometry property for Gaussian random matrices. 

We have the following estimates on the quantities 0™ ax , and 0™ ax , cn“ m . 


Theorem 1.4. Let A be an m x n random matrix with i.i.d. entries obeying M(0, 1). For 0 < [3 < 1. 


( 1 . 21 ) 

( 1 . 22 ) 

and 


(1 - min ( 
max (c^ 111 y 


3 — 2/3 


4(1 -P) 


, 1) < 9J ax < min 


7r 


-/T 2 


,-(3 z < wf < 2 In 


In 


1\ 2 


P 


1-/3’ 


(1-23) |(1 - < (1 - «min 

(1.24) < 2(1 - /3 )In 

where the absolute constant c g is defined in \3.15\) . 


Theorem O establishes the strong restricted isometry property for Gaussian random matrices for all 
/3 € [0,1) by (1 — (3) In y/bj < 1 for any (3 € (0,1) and by (11.31) for (3 = 0. As a direct consequence of 
Theorem P1 we have 


vr op 

6 - (1 — (3) 2 


< 2vr(ln2) 2 , 


, .min 

C 2 < ^ 

9 ~ In — 

111 1-/3 


< 2 + -—, 
“ In 2 ’ 


V 1/2 < /3 < 1 


and 


7T 

— < 


nmax 


(1 -f3f 


< 2vr(ln2) 5 


4< 




(1 - f3) In 


^ 2+ h^’ 


V 1/2 < /3 < 1. 


Thus, up to multiplicative constants, our estimates in Theorem 1 1.41 for 0™ ax , and #™ ax , are optimal 
as /3 —y 1 
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As an application of Theorem 11.41 and our analysis in Section U for proving Theorem 11.41 we have the 
following robustness properties of Johnson-Lindenstrauss lemma and restricted isometry property with a 
given erasure ratio 0 < (3 < 1. 

Corollary 1.5. Let 0 < /3 < 1 and 0 < a < ^(1 — f3) 2 hp with hp := min(| — ^(3 ,1 — /3). Let m, n,N G N 
such that m > p—p and m > p In ■ Let A be an m x n Gaussian random matrix with i.i.d. entries 

obeying Af(0, 1). For any given N points pi,... ,pn G with probability at least 1 — N(N — l)e~ am > 0, 

(1 25) HPj-Pkf < ^\\A T Pj ~ A T p k \\ 2 <u\\pj ~Pk\\ 2 , 

Vl<j,k<N and T C {1,..., m} with |T C | < /3m, 
where 6,ui G (0, oo) are positive real numbers given by 

(1.26) 9 := |(1 - (3 fhp + 2 a - 2(1 - f3)^irahp/3, u := ^2(1 - 0 - £) In 

Corollary 1.6. Let 0 < f3 < 1 and 0 < a < ^(1 — (3) 2 hp with hp := min(| — ^/3 ,1 — (3). Let m, n, s G N 
such that m > and s In < am — hr 2. Let A be an mxn Gaussian random matrix with i.i.d. entries 
obeying A^(0,1). For any 0 < e < 1, with probability at least 1 — 2(^|(p) s e ~ am > 0, 

(1.27) 0(1 — e)||u|| 2 < ^||Atp|| 2 < w(l + e)||n|| 2 , V s-sparse v G M n and T C {1,..., m} with \T C \ < /3m, 
where the positive real numbers 9 and uj are given in M.26\) . 



It is of interest to extend the main results in this paper from Gaussian random matrices to other random 
matrices such as sub-Gaussian matrices and circulant matrices (c.f. 53])- If A is the Bernoulli matrix, 
i.e., P(aj j fc = 1 1\Jm) = P(a^fc = —1 l\fm) = 1/2. Define 2-sparse vectors v\ := (1,1,0,..., 0) T G M n 
and V 2 := (1, — 1,0,... , 0) T G M n . Then either inf { A|| J 4 T? ; 1 || : T C {1,..., m}, \T C \ < m/2} = 0 or 
SU P (mil AtV 2 || : T C { 1 ,... ,m }, \T C \ < m/2} = 0. That is, for any 6 > 0, either P(O[ 0 iOO ] 5 l / 2 (A, ui)) = 0 or 
P(fl[ 0 t00 ].i/ 2 (A, V 2 )) = 0 for all m G N. As a consequence, the strong restricted isometry property for (3 = 1/2 
cannot hold for Bernoulli random matrices. This shows that the results and study for sub-Gaussian random 
matrices will be essentially different to Gaussian random matrices. We shall study random matrices other 
than Gaussian random matrices elsewhere. 

The structure of the paper is as follows. In Section 2, we shall provide some auxiliary results for the proofs 
of the main results in later sections. In Section 3, we shall study the robustness properties of Gaussian 
random matrices with arbitrarily erased rows for small distortion rates e —>• 0 + . In particular, we shall prove 
in Section 3 Theorem o and a few other results related to Theorem [□3 In certain sense, we studied in 
Theorem 11.11 the quantities /3 and for the case of small erasure ratios e —>• 0 + . In Section 4, we 

shall study the robustness properties of Gaussian random matrices with arbitrarily erased rows for a given 
corruption/erasure ratio 0 < j3 < 1. In particular, we are interested in the behavior of 0™ ax , w™ in and 0™ ax , 
w“ in when f3 —>• 1”. We shall prove in Section 4 Theorem 11.41 and other results related to Theorem 11.41 
We shall also show that our result leads to the establishment of the strong restricted isometry property for 
Gaussian random matrices. As applications of the main results in this paper for dimensionality reduction, 
in Section 5 we shall prove Corollaries 11.2111.31 and Corollaries 11.5111.61 


2. Auxiliary Results 


In this section we provide some auxiliary results that will be used in later sections. For y = {y\,... , y m ) T G 
M m , we define y^,... ,y^ m ) to be the nonincreasing rearrangements of y\,... ,y m in terms of magnitudes 
such that |?/(i)| > • • • > |y( m )|. Let m G N. For any 0 < 7 < 1 such that 7 m is an integer, we define 

(2.1) f 7 := {TC{l,...,m} : \T C \ = 7 m}. 


The following simple observation will facilitate our discussion in later sections. 
Lemma 2.1. For 0 < 7 < /3 < 1 such that both 7 m and /3m are integers, 


mm 

TeTg 


|7^y||Arxo|| 2 — 1 


< min 

TeT-v 


||^4t®o|| 2 — 1 


< max 

TeTy 


|T[ II^T x o|| 2 — 1 


< max 
T&T b 


|7^||^4t®o|| 2 — 1 
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Proof. Let fc 7 := 7 m and kp := (3m. By 0 < 7 < (3, we have fc 7 < kp and it follows from |?/(i)| > ■ ■ ■ > \y( m )\ 
that 


2 v ..,3 -+ "' 2 


. Pt^oII y (kp+l) 
nnn — 


T&Tp \T\ 


m — kp 

Pt^oII 2 y a) 


y (m) ^ yfk-,+ 1 ) + ■ ■ ■ + yfm) . \\A T X 0 \\ 

< -;- = mm 


m — k. 


7 


< max 


2 vf, h -h y 2 m _^ < yfi) d i- " ,2 


TeT 7 \T\ 

7 / 2 

y(m-kp) 


T£T 7 |T| „„ - n , 7 


m — fc 7 

Now the claim follows directly from the above inequalities. 


m — kp 


PtxqII- 

Tefp \T\ 


= max 


□ 


The following well-known concentration inequalities for the standard Gaussian/normal distribution (e.g., 
see [TO]) will be used later. 

Theorem 2.2. Let f : W l -A M be a Lipschitz function with Lipschitz constant 1 satisfying \ f(x) — f(y)\ < 
||x — y\\ for all x, y € M m . For i.i.d. standard Gaussian/normal random variables X\,... ,X m ~ A/"(0,1) and 
for all 5 > 0 , 

(2.2) ¥{f(X 1 ,...,X m ) <S + K[f(X 1 ,...,X m )]} > 1-e- 52 / 2 , 

(2.3) P{f(X 1 ,...,X, m )>-S + E[f(X 1 ,...,X m )}} > 1-e- 52 / 2 . 

As an application of the above result, we have the following result (also c.f. [ 13])- 

Lemma 2.3. Let yi, ■ ■ ■ ,y m be i.i.d. Gaussian/normal random variables obeying J\f (0,1). Then for all 
nonempty subsets S C { 1 ,..., m} and 5 > 0, 


(2.4) 

and 

(2.5) 


4[E»a> <{ + E ,/i^E»o-) 


1*1 u 


> 1 - e 


-5 2 |S|/2 


1 i f g4,>-5 + E 1 


ir w 2 \ > 1 _ e -<5 2 |S|/2 


Proof. Define Fs(x ) := dfj)- Then it is easy to observe that 

m m 

\f s (x) -F s (y)\ 2 < 5^(|s(j)| - \y(j)\) 2 < -1 yu)\f = \\ x \\ 2 + M 2 - 2 ^2\ x u)y(j)\ < \\ x ~y\\ 2 , 

j&S j =1 j =1 

where in the last step we used the rearrangement inequality 1 \ x jVj\ — J2JL 1 l x 0)2/L')l- Therefore, Fg is 
a Lipschitz function with Lipschitz constant 1. By Theorem 12.21 we have 




= r ,E»’ 1 ]< f vWtE Ei >1 


— e 


—<5 2 |S|/2 




jeS 


The inequalities in (12.51) can be proved similarly. 


□ 


The following result extends 0 Example 10]. We provide a proof here by modifying the proof of 
Example 10]. 

Lemma 2.4. Let y 1 ,... ,y m € K m be i.i.d. Gaussian/normal random variables obeying J\f(0, 1) and define 
ytpp ... ,y( m ) to be the nonincreasing rearrangements of yi,... ,y m in terms of magnitudes such that 12/(i) I E 
— I y(m) I ■ 

(i) For 1 < j < m and 1 < p < 2, 

( 2 . 6 ) +1 - E| ^ 1 and E M p < c pf2j^ C p(j + ]n j)’ 

^ j 
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where C p is a positive constant (e.g., C\ < C 2 < 2) depending only on p and given by 

r°° / _\ 1 _R 

(2.7) C p := p sup t p ~ l I e^ 2 ~ s2 ^ 2 ds < p (—) 2 . 

0<t<OO Jt V 2 / 


(ii) For 1 < k < m, 



(iii) Let 0 < 7 < 1 and m € N such that k := 7 m € N. Then 


(2.9) 



(I- 7 ) 


1 ~ 7 + 2 m 

1 + 777 


< E 


1 


1 


m — k 


E 4 ^ 


j=/c+l 


1_ 7 7+71 


' 2 - 


Proof. Define Ti(f) := y ^ J)°° e s2 / 2 ds. As shown in [3 Example 10], 

m-j 


p°° J fm\ r°° 

nyw\"=l uitoi >* 1/r }* = E (<)l 


m ~J / \ r 00 

7T \ - / m x ' 


E 

«=o 


(u(t)) m - £ (l - u(i))V p_ V 2 / 2 (-<Mi)). 


(i) Since e * 2 / 2 > 1 for all t € M, for p = 1 , by a change of variable x = u(t), as proved in [TJ Example 10], 

^mi a yig (?) - xYix= 

For 1 < p < 2, by a change of variable x = u(f), we deduce that 

m ~j /.00 I - / \ 

E|j/ 0 )| p = E J ^Pt r - 1 e‘‘ /2 u(t)( f J («(*))”-'-'( 1 - u(t))‘(-du(t)) 

< C„£ (+ f - x)'dx = C„ £ +1 ^ 1)!fl 

n \ /JO 0—c\ 


£=0 

= ^E^ = c p (l + e i)< Cp (i + fEx)=c p (i + i,^). 


£=0 


t=i+i 


J 


J 


It is easy to prove that if 1 < p < 2, then Cp < 00 . Indeed, define 


r c 

m :=t p -' J 


J?-s 2 )/2 ds = t p- 




-ts-sC^ds < t p - 


'f 


2 / 2 ds = .nt p ~ 1 


We also have 


POO POO 

f(t) = tP- 1 / e~ ts ~ s2 / 2 ds < t p ~ l / e~ ts ds = t p ~ 2 . 

Jo Jo 

a-? 


Therefore, C p = p sup 0<t<oo f(t) < p sup 0<t<oo min( v /f t p 1 ,t p 2 ) = p{ f) 2 < 00 . 
(ii) By (12. 6 p with p = 2, we have C 2 < 2 and 


E 


1 . p 

\ k E v {j) I - \ k ^ 

\ i=i / \ i =1 


iE e 4A 


/c m 


2 v+ 1 2 / , +A 1 \ / m lem 

E 7) -\/ 2 + 21 n T-\/ 21n_ r 


£=fc+i 


by ZXfc+i 7 < J™ ydx = In x- T bis P roves d+SD- 
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(iii) By the Cauchy-Schwarz inequality, we have 


v- m + l-j ^ 

E “VITi - Iwl £ 

j=k +1 


\ 


E 

j=k +1 


m + 1 — j 
m + 1 


N 


E 4>- 

j=k +1 


Therefore, it follows from the first inequality in (12.61) that 

1 


E 


\ 


- 1/2 


rr E 4 ) 


m — k 


j=k +1 



m _ 

v-^ 771 + 1 — 1 . 

E m+ i E i^ 

j=/c+l 
- 1/2 


E 

/=fc+i 


m + 1 — j 
771+1 


1 m 

1 

f m + 1 - j \ 2 

m — k ^ 
j=k+ 1 

^ 777 + 1 / 

j (777 — k + 1)(2t77 — 2k + 1) ^ 


m—k 


^ {m — k)(m + l) 2 ^ 


6(777 + l) 2 V 6 V 777(777 + 1) 

m ~ fc+1 by A; > 0. By k = 7777 , we proved the left-hand side of (|2.9D . 

On the other hand, it follows from the second inequality in (12.61) with p = 2 that 


since "" '"J" > — 

m +1 — m 


E 


1 


1 m 

rr E 4) £ 


m — k 


j=k +1 


N 


l 


m — k 


E E 4> s 


j=fc+i 


N 


m m 1 

E E) 


\ 


2 - 


2k 


m — k ' £ 
£=fc+i 


E4 + 


m — k ' l 

j=k +1 £=j 


2fe , 777 +1 

■ m ■ 


\ 


m i 

E E 


m — k ^ £ 

£=/c+l J=fe +1 


m — k k + 1 


E m 1 \ r 

7=fc+l 7 > J, 


m+1 lj„ _ U, m+1 


: dx = In tttt-. By k = 7777 , we proved the right-hand side of (12.91) . 


□ 


Let l+o : [—e _1 ,oo) —> [—l,oo) and W_i : [—e , 0) —>• (— 00 ,—1] be the principal and secondary real¬ 
valued Lambert VL functions such that Wo(x)e w °^ x ' > = x for all x > —e^ 1 and W-i(x)e w ~ 1 ^ x l = x for all 
—e~ l < x < 0 (see 0). Note that Wo is an increasing function while W-\ is a decreasing function. 

We shall also need the following auxiliary result in the proof of Theorem 13.41 of Section [3j 
Lemma 2.5. For any positive real number c„ > 0, 


( 2 . 10 ) 

and 

( 2 . 11 ) 


e 0 := max ■ 
y x>2 x - 1 


; ln(2x) — 1 c 2 ln^-l f c 2 ln4-l>0, 


0 < Pa < 


1 

9—2 


with 


- 1 




g— - ' i f c g ln '^ >1 ’ 

- C 2 Wo(-2e- 1 - 1 ^)>0, if c 2 In ^ < 1, 


b c 2 In -^ > 1, 

-W 0 (-2e- 1 - 1 / c «), i/c 2 ln^<l. 


Proof. Let t g := 2e 1 1//c s. If 0 < c 2 In -)= < 1, then 0 < t g < e 1 and therefore, both Wo{—t g ) and /3 g are 
well defined. Since Wq is an increasing function, it is also easy to prove that 


( 2 . 12 ) 


- W Q {-tg) < l 


To prove (12.101) . we define f(x) := 

fix) = 


c 2 g ln( 2 x)-l 


x — 1 


=> tg < ie - 1 / 2 

for x > 1. Then 


c 2 In 4 = < 1. 

9 \/e “ 


g(x) 


x(x — l) 2 


with g{x) := x + c 2 {x — xln(2x) — 1). 
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By g'(x ) = 1 — c 2 ln(2x), the function g increases on (0, and decreases on (^L-,oo). Note that f has 
the same sign as g on (l,oo). If tg > e-\ then c 2 In 2 > 1 and g'{x) < 0 for all x > 1. Hence, f'{x) < 0 
for all x > 1 and / decreases on (l,oo). Therefore, e g = max x >2 f(x) = /(2) = f(l//3 g ), since (3 g = 1/2 by 
c 2 In -^= > c l In 2 > 1. Since g' has only one real root on (0,oo), if t g < e 1 , then g has at most two real 

roots on (0, oo) given by x\ := _ w j < 1 < _ w ^_ t y =: X 2 ■ Thus, / decreases on (x 2 ,oo) and increases 
on ( 1 , 0 : 2 ), from which we have e g = max x > 2 /(x) = /(max(2, X 2 )) = f(l/(3 g ), since max(2,a:2) = l//3 g by 
(12T21) . □ 


3. Gaussian Random Matrices under Arbitrary Erasure of Rows for Small e 

In this section we study the robustness property of (normalized) Gaussian random matrices under arbitrary 
erasure of rows for small 0 < e < 1. At the end of this section, we shall prove Theorem ll.il 


3.1. A lower bound for To provide a lower bound for (3™ x in (11.61) . we first prove the following 

result. 


Lemma 3.1. Let A be an m x n random matrix with i.i.d. entries obeying AA(0, 1). For 0 < a < 1 and 
0 < e < min(l, ^7^), if 

(3.1) 0 < P < 1 1 + ^ e and 0 < (3ln ^ ^ 1 - , 

then 

(3.2) P(n e ^) > 1 - 3 e -“h 2 / 4 -e 3 /6)m, VmeN. 


Proof. Set y := Ax o- Since each entry in A is an i.i.d. random variable obeying A/"(0,1) and since ||a:o|| = 1, 
a simple calculation leads to y g ~ jV(0,1) for every j = 1,... ,m and all y\..... y m are independent. 

Let m € N be arbitrary but fixed at this moment. Define 7 := \_f3m\/m. Then 0 < 7 < j3 and 7 m € N. If 
7 = 0, then (3m < 1 and consequently \T C \ < (3m implies that T c is the empty set. Therefore, if 7 = 0, then 
0 £ji g = D £! o an d the claim in (13.21) is trivially true by (11.3|) . Thus, in the following discussion, we assume 
7 > 0 . 

By Lemma eh we conclude that 


P(fi 6ij8 )=P jl-e< 


\\VT\? 

\T\ 


< 1 + e V \T C \ < (3m >=P<1 — e< min 


WtI 


< max 


hrf 


rer 7 \T\ Ter-, \T\ 


< 1 + e 


where T 7 is defined in m- Recall that y^,..., are nonincreasing rearrangements of yi,...,y m in 
terms of magnitude such that 1 2 /( 1 )I > 1 2 /( 2 )I > ■■■ > |z/(m)|- Let T £ T 7 and k := \T C \ = 7 m. Then 
\T\ = m — k. Since || 2 /t || 2 = ||y || 2 — ||?/T c || 2 , we observe that 

(3.3) |M| 2 - (y 2 1} + ■■■ + y 2 k) ) < ||y T || 2 < ||y|| 2 - ( yf m _ k+1) + ■■■+ yf m) ). 

Using the above inequalities and noting that k = 7 m, we can easily deduce that 

m ,, . . WutW 2 ^ WutW 2 ^ , , 

P s 1 — e < mm ——— < max — < 1 + e 


tst 7 \T\ TeT 7 \T\ 

\ 2 -(y 2 a) + --- + yf k) ) 


= P<1 -e < 


2 /( 1 ) 


+ ••• + 


m — k 
,2 „ 112 


< 


- (y 2 m -k+ i) ^ yf m )) 


m — k 


< 1 - e 


+ ••• + 


y{m) 


y (k) < ]}yf _ (1 — T)(l - e) and y(m-k+l) 

k ~ k 7 k 

> P (Eo r\E 1 r\E 2 ) = l - P(£0 u E{ U E c 2 ) > 1 - P(Eg) - P (El) - P(£§), 


> 


(1 - 7 )(l + e) 


7 
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where 

(3.4) 

(3.5) 

(3.6) 
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Eq := < 1 — yfae < --< 1 + y/ae f , 

m 


Ei : = 


E, : = 


'yfa + --- + yfa < 1 _ e+ Wa e ' 


k 


7 


y(m) H h ^(m-fe+l) . 1 1 - \/a 

- i >l + e-e 


k 


7 


Since i?o = {|;^-||y|| 2 — 1| < y/ae], it follows directly from (11.31) that P(£o) > 1 — 2e ( ae2 / 4 « 3/2 e 3 /6)m_ 
Thus, by 0 < a < 1, we have 

p(£C) ^ 2 e -(ae 2 /4-a 3 / 2 e 3 /6)m __ 2 e -«(e 2 /4-£ 3 /6)m e -(l- v /a)ae 3 m/6 ^ 2 e —“(e 2 / 4 —e 3 /6)m 
Next we estimate P(-Ei) and P(i?2). By (|2.8p of Lemma [2~T1 and noting that k = 7m, we have 


E, 


1 




E7, - \ 2ln 


7 


For 5 > 0, it follows from (I2.4p of Lemma 12.31 with S = {1,..., A;} that 
(3.7) P 


N 


1 




3 =1 


1 




rE^h 1 -^ 2 - 


3 = 1 


Take 

(3.8) 

We claim that 

(3.9) 


5:— \ 1 — e + 


1 - >/« 


7 


e — W 2 In 


/ ae^ 

- V "27" > ■ 


Then it follows from (13.71) and the above inequality that 
P(F^) = P 


\ 


jE^W 1 -^ 


1 - \[0L 


e > < e 


—5 2 7m/2 ^ ae 2 m/4 ^ a(e 2 /4—e 3 /6) 


< e 


l=i 


7 


1 — \fot 1 — \fot , . 

1 + e-e < 1 + e--—e < 1 + e — (1 + e) = 0, 


On the other hand, by the first inequality in (13.ip and the fact that 0 < 7 < /3, we have 


which yields 

P(£ 2 ) =P' 


7 


P 


y\m) 4 f yfm -k+l) > 1 + e _ 1 - \/« c > 




k 7 

That is, P (E 2 ) = 0. Putting all estimates together, we complete the proof of (13.21) . 
We now prove (13.91) . By our assumption 0 < e < 1 ~^° , we observe that 


= 1. 


(3-10) fe,a> 0 with f £>a := y-(^1 - 

Now it is straightforward to check that 

(3.11) 2x 2 + 2Vae 2 x < 2/ 2 Q + 2 a/ ae 2 / ejCt = (1 — \/a)e — ^§-, 


for all a; € [0, /e, Q ]. 
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Note that the function xln| is an increasing function on the interval (0,1]. Since 0 < 7 < /3 < 1 , by the 
second inequality in (13.11) . we have 


0 < \/7 ln l < \/pl n § < fe,a- 


Consequently, plugging x = ^7 In ^ into the inequality in (13.111) , we deduce that 

2 7 Inf + 2j^.fchJ-< (l-,/5)e-^. 


' 7 ' ' ' V ' 7 — v v _/ 2 

Since 7 > 0, dividing 7 on both sides of the above inequality, we obtain 


2 In - + 2 \/tt-- t /2 In - < 

7 V 2 7 V 7—7 27 


Hence, by 1 — e > 0, we conclude that 

2 


21 nf+ ^ 


= 2 l n £ + 2 ^ V ^I 


2 y 7 7 


That is, we proved 


2 In - + 

7 


^7 


< l/l — 6 + 


1 - \/a 


7 


□ 


which is simply the inequality in 

The following result establishes a lower bound for in (11.61) . 

Theorem 3.2. Let A be an m x n random matrix with i.i.d. entries obeying jV"(0,1). For 0 < a < 1 and 
0 < e < min(l, ±^) ; i/ 

(1 - Va)e 


(3.12) 

then 1.9, HI) holds. Consequently, 

(3.13) 


o < /?< 


16 In 


/omax ^ 
Pe,a ^ 


e 1 — \[a 


In - 32 


(l-v^)e 


VO < e < 


1 - va 


Proof. We first show that (13.121) implies (13.11) in Lemma 13.11 Since 0 < e < 1 and 0 < a < 1, we have 
0 < (1 — Ja)e < 1 . The first inequality in (13.11) follows from (13.121) . since 

0 < D < := / ~ J> C < (1 7.^ 5)e < (1 ~ ^ < (1 ~ ^ 


16 In 


(i-V5)e 


16 In 4 


1 + e 


Let f( i0 be dehned in (|3.10D . By 0 < e < 1 7^ , we have ^ < 1 and hence, / e>a > 0 and 


f z =- 

Je,a 2 


1 — Jot — I >4 


1 - Jot — 


1-V7 \ _ 


(3.14) 

A basic calculation shows that 

ln(4ez) < (8 — 4 \/ 2 ) 2 :, V z > In 4, 
from which it is straightforward to deduce, by setting z = ln(l/x), that 


9-4\/2 


16 


(1 - Ja)e. 


x , 4eln(l/x) 9 — 4j2 
In-< --- x , 


41n(l/x) 


x 


VO < x < -. 

4 


Plugging x : = 


_ (j-Vj)£ 


<1/4 into the above inequality, by (13.141) . we conclude that 


ie,aln i e ,a 41n(l/x) 


x , 4eln(l/x) 9 - Ay/2 9-4 J2, „ ,, 

In-< -_2_ x =-_I_(1 - Ja)e < f\ c 


x 


4 16 

Since /31n-| is an increasing function on (0,1], the second inequality in (13.11) follows from (13.121) by noting 
that 0 < fj In | < t^ a In < ff a . 
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Since t e)Cl < — and t e , a \n < f 2 a , there must exist <5 > 0 such that (13.11) holds for all 0 < (3 < 
t e a + 5. Note that e < 1 ^ < 1 ^ by 0 < a < 1. We have In j > In 1 _ 4 ^ and therefore, 


c« ax > k* + 5 > 


(1 - yg)e > (1 - y/g)e 
16 ( ln 7+iW7^)“ 321n 7 


This proves (13.131) . 


□ 


3.2. An upper bound for /3™^. We now show that the order = 0(e/ln^) for small e given in 

Theorem 13.21 is optimal. To do so, we recall a well-known inequality on order statistics (e.g., see item (ii) 
of [2 Example 10] or see [2 Lemma 3.3.1]): there exists an absolute positive constant c g (depending only 
on the Gaussian/normal distribution jV(0,1)) such that 

(3.15) 


c 9\J ln lT - E M> V 1 < j < m/2. 


We first estimate the quantity P(fi eii g). 

Lemma 3.3. Let m € N and 0 < /3 < 1/2 such that /3m € N. Let e > 0 and 0 < e < 1. If 


5 := c g f^ - y/(l — e)/3 + e + I > 0, 


(3.16) 
then 

(3.17) P(n £i ^) < e - e ' 2m/4 + e~ s2m/2 . 
Proof. Let y := Ax o and k := /3m. By Lemma 12.11 and (13.3|) . we have 


P(O e> ^) = P 
= P 


< max | ttttt 


[TeTp 


\T\ 


\\y T \\ 2 - 1| < e) < pjmin ||y T || 2 > (1 - e)(l -/3)m 

J l tet p 


’{iMI 2 - (vfi) + • • • + vfk)) > (i -e)(i-/3)m} 

< F &y\\ 2 > 1 + e} + P{2/(1) + • • • + yf k ) < ym}, 
where 77 := (1 — e)/3 + e-\-e>0 since 0 < e < 1. It follows directly from (11.21) that 


(3.18) 

On the other hand, 


P{^||y|| 2 >l + e}<e- e ' 2m / 4 . 


’ {y\i) + ■ ■ ■ + yfk) < v m } 


= r 


\ 


eE»m 


-E 


3 = 1 


\ 


1 ^ 

h E »W 


3 = 1 


< J--E 

P 


\ 


iE»&> 


3 =1 


Since k = (3m < m/2, it follows from the inequality in (13.151) and |i/(i)| > ■ ■ ■ > \y(k)\ that 


E \ - E fe)D ^ c 9\/ ln x = ° 9 \[^Tr 
N j =i 


Therefore, 


P 


{»(d +" ■+»« < f™}= p I ^ E ~ E £ \/f - E y 2 m 


< P 


\ 


:E>m 


-E 


3 =1 


\ 


rE«, 


3 =1 


A 


E 4) 


-E 


3 =1 


\ 


1 

iE»« 


< 


< -- 


3 =1 


/5 

'v^ 


- c„Wln — 
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By our assumption 5 > 0, applying (12.51) in Lemma [2731 with S = {1,..., k}, we get 
’ {»fi) H-M/(») < f™} < f • 


p- 


N 




J =1 


\ 


it £4) 


1=1 


<_ — \< e ~ s2k /W) = g— <5 2m /2 

V/3 


Combining the above inequality with ()3.18l) . we conclude that (|3.17l) holds. 


□ 


To provide an upper bound for in (11.6|> . here we introduce a related quantity. For e > 0, a > 0 and 
C > 0, we define 

(3.19) (3™* c := sup{0 </3< 1 : P(D £j/3 ) > 1 - Ce ~ a ^^for sufficiently large m £ N}. 

If the above set on the right-hand side of (11.61) is empty, then we simply define := 0- Trivially) 

^max: < f or J £ > Q an d a > 0. 

Theorem 3.4. Let c g and e g be the absolute positive constants defined in \S.15\) and \'2.10\) . Then 


(3.20) 


/omax ^ 
Pe,a,C < 


(i + e 3 ) e 

C 9 l n 


V0 < e < min(l, e g ),a > 0 ,C > 0. 


Proof. If = 0, the claim is trivially true. Hence, we assume > 0. We first prove that 

(3.21) f e {(3) := Cg^J/3 In | - y/(l-e)fi + e <0 V 0 < (3 < min(±, 

By the continuity of the function f e , it suffices to prove (13.211) under the extra assumption that f3 is a rational 
number. Suppose that (13.211) fails for some rational number (3 such that 0 < (3 < min(^, Then there 

exists e > 0 such that 

<5 = Cg^Jfi In -T' - a/(1 - e)(3 + e + e > 0, 
where 6 is also defined in (13.161) . Consequently, by Lemma [3731 

(3.22) P(ft £i 0) < e" 6 ' 2 " 1 / 4 + e ~ s2m/2 
provided /3m £ N. On the other hand, by the definition of [3™(ffi c and 0 < (3 < 

(3.23) P(fi £ ^) > 1 - Ce- Q(e2/4 - e3/6)m for sufficiently large m £ N. 

Consequently, combining (|3.22l) and (13.23[) . we have 

1 - Ce-“( e2/4 " e3/6)m < P(H £)/3 ) < e" £2m/4 + e~ s2m/2 

for sufficiently large m £ N satisfying (3m £ N. Since (3 is a rational number, there are infinitely many 
sufficiently large m £ N satisfying (3m £ N. Letting such m go to oo, we deduce from the above inequality 
that 

1 < C'g-«V 2 /4- e 3 /6)m + e -Pm/A + g -<5 2 m/2 

which is a contradiction. Therefore, (13.211) must hold. 

Define a function 

F e (P) ■= c g/31n§ - ((1 - e)/3 + e), (3 > 0. 

Then it is trivial to see that f e ((3) and F e (f3) have the same sign on the interval (3 £ (0,1). As a direct 
consequence of (I3.2ip . it is straightforward to see that 

(3.24) min(i, < inf {° < P < 1/2 : Ffi(3) > 0} =: (3 e . 

If the above set on the right-hand side is empty, then we simply define (3 e = 1/2. Let (3 g and e g be defined 

as in Lemma [2751 Since 0 < (3 g < 1/2 and e g > 0, we deduce that 

F e ((3 g ) = c 2 (3 g In ^ — ((1 - e)(3 g + e) = (1 - (3 g )(e g - e) > 0, V0 < e < e g , 

where we used c 2 In ^— 1 = e 9 (^— 1) by (12.101) . Since lim,g^ 0 + F e (/3) = —e < 0, F e must have a real root 
inside the interval (0 ,/3 g ). Hence, 0 < (3 e < (3 g < 1/2 and F e ((3 e ) = 0. For 0 < e < e g , 

e = F e ((3 e ) + e = (c 9 In -gj — 1 + e)(3 t > ( c g In — 1 )(3 t = e 9 (^ ~ 1 )(3e (3 tgfie, 
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by 0 < (3 g < 1/2. It follows from the above inequality that f3 e < f- for all 0 < e < e g . Hence, for 0 < e < e g , 
it follows from F £ (/3 £ ) = 0 and 0 < /3 e < f- that 


Pe = 


e + (l-e)& < e + (!- e )t 


=?ln#- 


c 91 11 


By 0 < /3 e < /3 g < 1/2, we conclude that 


< 


(1 + e g 1 )e 

4^ ' 


(3.25) 




(1 + e g 1 ) e 

c g In ~T~ ’ 


VO < e < min(l, e g ). 


This proves (13.201) . 


□ 


3.3. An estimate for We now study the relation of the quantity /3 ™q X in (jl. 8 p using fi £ii g in (11.7|) 

with a uniform normalization factor —. 

m 

Similar to Lemma 13.31 and Theorem 13.21 we have the following result on the lower bound of 

Theorem 3.5. Let A be an m x n random matrix with i.i.d. entries obeying jV(0,1). For 0 < a < 1 and 
0 < e < min(l, 1 ~^ ) ; if 0 < /3 < 1 satisfies the second inequality in m, i-e., 

(3.26) 0 < 0 In ^ | (y/l-y/a - , 

then 

(3.27) P(O e ,/ 3 ) > 1 - 3e- Q(e2/4 - £3/6)m , VmG N. 

Consequently, under the same conditions as in Theorem 13.21 all the claims in Theorem 17.71 hold with 
being replaced by P'ptfp- 


Proof. Let 7 := [Pm\/m and k := 7 m. Then P(fl £)( g) = P(fl £i7 ). By ||y-r|| < ||y|| and 0 < a < 1, it follows 
from (|3.3I) that 

P(H £i7 ) = P |l - e < min ±\\y T \\ 2 < max i||y T || 2 < 1 + e j 

> P {(1 — e) 7 Ti < \\y\\ 2 - (y 2 1} + ■ ■ ■ + y 2 {k) ) and ±\\yf < 1 + v^e} 

>^{y 2 i) + ---+y 2 k)<iX-Va)em and |^||y || 2 - 1 | < Vae} 

= P(E 0 DE 3 )> 1 - P (E c 0 ) - P (E c 3 ), 

where E 0 := {|^||y || 2 - 1| < y/ae} as in (J33D and E 3 := {y 2 {V] H-h y 2 fc) < (1 - y/a)em}. By (JOJ and 

0 < a < 1 , we have 

P(L?o) < 2e - ( ae2 / 4- “ 3/2e3//6 - )m < 2e -a( ' e2 / 4-e3 / 6 ) m . 

Recall from (13.71) that the following inequality holds for any <5 > 0: 


(3.28) 

Set 


P 




£VA 5 + E 


N 




6: = 


Since the function p In is an increasing function on (0,1], by 0 < 7 < P < 1, we deduce from (13.261) that 


0 < 7 In - < p In J < J 
7 P 2 



2 
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Since 7 > 0, dividing % on both sides and then taking square root on the above inequality, we see that 




/ (1 - y/oQf _ / ae 2 
V 7 V 2 7 ’ 

from which it is trivial to see that 5 > ^ j > 0 holds. By the dehnition of the set E%, it follows from (I3.28P 
and 5 > \! c 3f— > 0 that 


P (Eg) = P < 


\ 




> 


3 =1 


(1 — \/a)e 

7 


\ 




f=i 


> S + ^ / 2 In — 

7 


< < e -ae 2 m/4 < e -a(e 2 /4-e 3 /6)m. 

Therefore, P(fi £)/J ) = P(S\ 7 ) > 1 - P(£§) - P(£§) > 1 - 3 e -«L 2 / 4 - e 3 / 6 )m. Thig proveg 

It has been proved in the proof of Theorem 13.21 that (13.121) . combined with 0 < a < 1 and 0 < e < 
min(l, 1 4 ^ ); implies the conditions in ()3.1j) with < being replaced by <. Therefore, all the claims in 
Theorem 13.21 hold with Pf 1 ^ being replaced by /3, 


d max 
J e,a ' 


□ 


To provide an upper bound for we define P™ x c as ai dSH]) with fl £)/ g being replaced by Cl €i p. 

Trivially, P™ x < f° r all e > 0 and a > 0. 

Theorem 3.6. Let c g be the absolute positive constant in H3.15\) . Then 


(3.29) 


omax ^ 
Pe,a,C — 


< 


—CgWi(—e/ ( 2 c|)) 2 c? in —^ 


V0 < e < min(l, c 2 In 2 ), a > 0, C > 0. 


Proof. If p^ c = 0, the claim is trivially true. Hence, we assume Pf^^c > 0- We P rove that 
(3.30) g e (/3) := c g ^ p In | - Ve < 0 V0</3< min(±, P™* c )- 

It suffices to prove (13.301) for rational numbers p. Suppose not. Then there exists e > 0 such that 5 := 
c g yj p In — \/e + e > 0. Let y = Aco and k := /3m. Then 

P(^e,/ 3 ) <P{mm ||y T || 2 > (1 - e)m} < P{^ ||y|| 2 > 1 + e} + Pjy^ + • • • + yf k) < rjm} 

with 77 := e+e. The same argument as in Lemma [373] yields P(S4 e ,/3) < e ~ e2m / A -\ _ e -< 5 2 m/ 2 . gj nce 0 < p < ppppj, 
the definition of P'pffc implies P(fl £ ,/ 3 ) > 1 — 3 e - a ( e2 / 4 ~ e3 / 6 ) m an d the same argument as in Theorem 13.41 
leads to a contradiction. Therefore, (13.301) must hold. 

Note that g e is an increasing function on (0, |) and | < |. By < In2 and the simple fact — W-\{x) < 
—21n(— x) for all x € (—e _ 1 ,0), it is easy to conclude from (13.301) that (|3.29[) must hold. □ 


3.4. Proof of Theorem II. 11 We are ready to prove Theorem 11.1 


Proof of Theorem \1.1\ The left-hand inequality in (11.91) follows directly from (13.131) of Theorem 13.21 Since 
0 < e < 4e 2 , we have y/e < 2e g and therefore, In > In ^ | In i. By Theorem 13.41 we have 

(l + e q 1 ) e „ (l + e < ? 1 ) e 2 + 2 e„ e 


P™ x < A max 


e,a, 3 


< -- 2 —— < 

r 2 lv, l c 2 l n 1 

c g m ~ 2 c 3 m e 


cje g ln i 


This proves the right-hand inequality in (jl.9p . 

The left-hand inequality in (11.101) follows directly from Theorem 13.51 and (13.13j) of Theorem 13.21 Since 
e < 2 ^ 7 , we have ln \ > ln(2c 2 ). Note that 0 < min(c 2 ln2, -^-) < 1. Now the right-hand inequality in (11.101) 
follows directly from Theorem 13.61 □ 
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4. Gaussian Random Matrices under Arbitrary Erasure of Rows for given 0 < (3 < 1 


In this section we study the robustness property of Gaussian random matrices with arbitrarily erased rows 
for a given corruption/erasure ratio 0 < ft < 1 with presenting the proof of Theorem 11.41 

4.1. Estimate 0™ ax (a) and u ;™ in (a). To prove Theorem 11.41 we first estimate 0™ ax (a). 

Lemma 4.1. For 0 < f3 < 1 and 0 < a < ^(1 — /3) 2 /ig with hp := min(| — ^/3,1 — /3), 

(4.1) 0 < |(1 - P)hp + - 2^/irahp/3 < 0™ ax (a) < min - 1 )- 

Proof. Define 7 := [/3m\/m and k := 7 m. Let y := (yi,..., y m ) T := Ax 0 . Since 7 m = |_/3mJ, by Lemma [2TT1 
and (13.311 . for 6 > 0, we have 


(4.2) 


n^oo],y) = p {“in j^WvtW 2 > 0} = p|^ y\ 3) > V0J . 


By the left-hand inequality in (12.9jl and 0 < 7 < /? < 1, we have 
E 


\ 


1 

m — k 


E yb 

j=k +1 


> 


(I- 7 ) 


!“7+ 2 ^ 


2m 


> 


1 + 


1 — 


'( 1 -/ 3 ) 


1 - P + 27; 


2m 


m 


> \ fV( 1 ~P)\ inf 


1 - /? + 


0<a:<l 1 + X 


= \Iq\ /(!-/3)V 


Therefore, for 0 < /3 < 1, by (14.2|) and (12.511 of Lemma [2731 with 6 := \f\\J (1 — /3)fyg — \/$, if <5 > y > 0, 
then we have 


P(^[0,oo],/3) = 


= r 


IN 


l 

m — k 


E y 2 u) 


-E. 


j=k +1 


1 


1 


m — k 


E 4) a ^ 


-E. 


j=fc+i 


'N 


l 


m — k 


E 4 


j=*:+i 


> 


.N 


l 


m — k 


E 4) - E 


N 


1 


m — k 


m I — 

E y (i) - v|\A 1_ ^ 


j=k +1 


> 1 — e = 1 — e i -' 3 > 1 — e 


j=fc+i 

> 1 _ g—<5 2 (m—fc)/2 > j 

since >lby 0 < 7 </ 3 <l. This shows that if \/0 < y^"-y/(l — P)hp — yJ jpp, then 
(4-3) P(%oo],/j) > 1 - e-“ m 

for all m € N. Since 0 < a < ^(1 — f3) 2 hp, we have ^J \\/(1 — P)hp — \Jjpp > 0. Consequently, by the 
definition of 0 ™ ax (a), we conclude that 

»"“(«> > ja-nhf - ^E) = l a - n)b + - 2yEv3 > 0 . 

This proves the left-hand side of (14.111 . 

We now estimate the upper bound for 0™ ax (a). By the second inequality in (12.611 with p = 1, we have 


(4.4) 


1 1 PP /I , m \ pi r, m pP 

^. E t % £ e i^+d u v 2 (fu + ln urr) - v 2 1,1 t = v 2 


'Uni, 

2 7 ’ 


where we used the basic inequality < J n (1 4 . for all k > 0. Suppose that (I4.3|l holds for sufficiently 
large m € N. For convenience, we only consider the case that /? is rational, since the general result follows 
from the fact that the rational numbers are dense in R. We assume that m € N is sufficiently large and 
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satisfies m(3 € N, that is, we have 7 = /3. Note that k = 7 m = /3m. By (H2D and (14.41) . applying (12.41) of 
Lemma 12.31 with 6 := \[& — In ^ > 0 , we have 


E(^[0,oo]„a) — 


A 


1 


m — k 


E 4> 


— E 


j=k +1 


A 


1 


m — k 


E 


j=k+l 


\ 


1 


m — k 


E 4> 


j=fc+i 


< 


A 


1 


m — k 


E 4 - E 


j=fc+i 


N 


l 


m — k 




j=fc+i 


7T . 1 


< e 


—5 2 (m—fc)/2 g—5 2 (1— /3)m/2 


Consequently, if \/$ > In ^ and if (14.31) holds for sufficiently large m € N, then the above inequalities 
imply 

1 _ < P(f2 [W ) < e -5 2 (i-A)W2 ) 


which cannot be true for sufficiently large m since a > 0 and 6 > 0. This proves that d^ ax (a) < -|(hi ^) 2 
Also, it is trivial to see that 


E. 


A 


1 


m — k 


E yl) 


< 


j=k +1 


\ 


1 


m — k 


E E »i 


U) - 


j=k+1 


\ 


1 


m 


EHi = L 


7=1 


The above same argument shows that 0“ ax (a) < 1. This proves the upper bound of (14.11) . 

We next estimate w™ m (a). 

Lemma 4.2. For 0 < /3 < 1 and a > 0, 

(4.5) “ax (<* In - 2 -, If?) < < 21 n ^ ^ In T 


□ 


~P 

Proof. Define 7 := [/3mJ /m and A; := 7 m. Let y := (yi,, y m ) T '■= Axq. Since 7 m = |_/3mJ, by Lemma [2~T1 
and (13.31) . for cj > 0, we have 


(4.6) 

By (12.81) . we have 


p ( fi [o,H^) = p {™^l|yT|| 2 <^} =p|^ E^a^^J 


E 


\ 


m—k 


rr E 4> 


m — k 


3 = 1 


< W 2 In 


em 
m — k 


= . / 2 In ■ 


1-7 


By (14.61) and the above estimate, applying (12. 4|) of Lemma [2731 with 6 := y/uj — ./2lnjf— > \ jffz > 0, we 


have 


p (^[o,HA) = P 


{ 1 m—k 1 m—k 1 m—k 

\ TTUt E 4) - E 4) s twi E 4; J 


(4.7) 


> 


1 


1 


m—k 


m — k 


E 4> - E 


j=i 


\ 


1 m—k 1 I 

7=1 J 


> 1 - e - 5 2 (m-fc )/2 > _ e -“T=^ = ! _ e -« 


If \/w > + y 2 In ^, then we have ^ 2 ln r=^ - y 7=77 + V ' 2 ln 1=7 by 0 - 7 - ^ < 1 

and the above inequality shows that 

(4.8) m [0M ,p) > 1 - e~ am 



































































18 


BIN HAN AND ZHIQIANG XU 


holds for all m £ N. Therefore, we proved 


O) < 


2 a 


2 In ■ 


e 2 a / a , e 

= 2 In --- + --- + 4, /--- In ■ 


1-/3 1-/3 V 1-0 1-/3 


1-/3 V 1-/3, 

This proves the right-hand side inequality in (14.51) . 

Without loss of generality, we assume that (3 is a rational number and m is sufficiently large satisfying 
/3m € N. Thus, 7 = /3 and k = /3m. We consider two cases of /3. Suppose that 1/2 < /3 < 1. Then 
m — k = (1 — /3)m < m/2 and by (13.151) . we have 


E 


\ 


1 m-fc I -9- / 9 

^ E £ s = c »v ln i7s' 


By (14.61) and the above inequality, applying (12.51) of Lemma 12.31 with 5 := c g ^j hiy^j — y/u} > 0 and 
S = {1,..., m — A;}, we have 


P (^[0,o;]„3) — P 


N 


m—k 


m — k 


3=i 


m—k 


^ 11 u r\j lit rv 

— E 4) - E «w £ vC 


-E 


3=1 




m—k 


rr E »?o 


m — k 


3=i 


< 


\ 


1 


m—k 


m — k 


E 


- E 


3=1 


1 


m—k 




m — k 


3=i 


■ c »'/ ln T^7 


\ 


1 


m—k 


m — k 


E 4) - E 


3=1 


\ 


1 m—k 

— k E 4) £ 


D —5 2 (m—k)/2 _ — i5 2 (1— ft)m/2 


< e “ v "'' ' v/ “ = e 
Consequently, if (14.8j) holds for sufficiently large m € N, then 

l-e-“<P ( fi [M , /J )<e- f2 (i-« ra /2, 

which cannot be true when m is sufficiently large. This proves that cu™ m (a) > c 2 ln^Ej provided that 

1/2 < p < 1. 

Suppose that 0 < /3 < 1/2. By (12.61) . we have 


1 ^ , HFk + l Rk R n 

E \m-k ^ y (3) - — y 2 m + 1 “ \ 2 m ~ \j 2^' 


The above same argument shows that w™ m (a) > |/3 2 provided that 0 < /3 < 1/2. This proves the left-hand 
side inequality in (14.51) . □ 


4.2. Estimate 0™ ax (a) and cj™ in (a). As a direct consequence of Lemma 14.11 we have 
Corollary 4.3. For 0 < /3 < 1 and 0 < a < ^>(1 — /3) 2 /ig with hp := min(| — i/3, 1 — /3), 


7 r 


7T / 1 


(4.9) 0 < -(1 - Py/ip + 2a - 2(1 - /3)yJ-Kahp/3 < OJ^ia) < (1 - /3) min ^ln-J > 1 

Proof. Define 7 := \_/3m\/m and k := 7 m. Let y := (yi,... , y m ) T := Axo- Then 0 < 7 < /3 < 1. By the 
definition of fi[ 0 )OO i £, we have 


p (^[0,oo]„a) = I 1 


.N 




“ P (°[ t ^-,oo],/3) ^ P (°[« ,oo],fi)> 
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where we used by 0 < 7 </ 3 <l. Consequently, for all 0 < 9 < (1 — /3)0™ ax (a), we have 

< 0 ™ ax (a) and by the definition of 0 1 p ax (a), we have 

1 _ e -«" < P(fi [ _ 4 _ i0o] ^) < P(0 [e>oo]i/3 ), V m G N. 

For 0 < j3 < 1 and 0 < a < ^(1 — j3) 2 hg, it follows from the above inequality and Lemma 14.11 that 

9f ax (a) > (1 - P)9ft ax (a) > |(1 - P) 2 hp + 2a- 2(1 - P)y/irah fi /3 > 0 . 

This proves the left-hand side inequality in (]4.9I) . 

Note that we proved the upper bound of #™ ax (a) in (14.11) of Lemma 14.11 bv assuming that f3 is rational 
and m is sufficiently large satisfying /3m G N. For such f3 and m, we have 7 = (3 and consequently, the same 
proof of Lemma 14.11 yields 

9f^(a) = (1 - P)0^(a) < (1 - (3) min(|(ln I) 2 , 1 ). 

This proves the right-hand side inequality in (14.91) . □ 


With the help of Lemma 14.21 we have the following result. 

Corollary 4.4. For 0 < (3 < 1 and a > 0, 

(4.10) (1 - p) max (c 2 In y-^, |/3 2 ) < w” m («) 
and 

(4.11) P(f = l[o,a;], / 9 ) > 1 — e _am , Voj> (^2(1 — 7) In ^ ' + \[2 , mGN with 7 := [f3m\/m. 


Proof. Define 7 := \j3m\jm and k := 7 m. Let y := (yi,... , y m ) T := Axq. By the definition of D[ 0 A ,] )( g, 


F (^[0,o;],/3) = P 



m^u)- 


Note that we proved the lower bound of w™ m (a) in (14.51) of Lemma 14.21 bv assuming that f3 is rational and 
m is sufficiently large satisfying /3m G N. For such (3 and m, we have 7 = ft and the left-hand side inequality 
in (14.101) follows directly from the same proof of Lemma 14.21 


As proved in g3D, if 5 := - ^2 In ^ > 0 , then 

(4.12) 3P(0[o 5 £ ^,] 5 ) a) = IP(O[ 0 i _i£_] i/ 3 ) > 1 — e -52(m- ^/ 2 > 1 — e -am , VmGN with 7 := \J3m\/m. 

This proves the inequality in (14.111) . □ 


To estimate the upper bound of cj™ in , we need the following lemma. 

Lemma 4.5. Let xq G M n with ||xo|| = 1 and m G N. Let A be an m x n Gaussian random matrix with 
i.i.d. entries obeying jV(0,1). For 0 < f3 < 1 and a > 0 , let be the smallest u > 0 such that 

P (^[0,w],/3) = P I SUp —\\A T X 0 \\ 2 <Uj\>l-e~ arn . 

Then uj/s tTn (a) > 0 for all a > 0 and u}p !m := lim ct _ >0 + = 0. 


Proof. Suppose that = 0 . Then P(f2[o, 0 ] ,p) > 1 — e " m , which is a contradiction to P(fi[o,o],/ 3 ) = 0 

and a > 0. Therefore, we must have up >m {a) > 0. Note that ujp^ rn (a) is an increasing function of a. By 
P (^[ 0 ,o; / 3 , m (a)],/ 3 ) > 1 - e~ am , we have < e"" m Then 


p(n 


[0 ,Up,m],P' 


lim P(D 

a—>0+ 




< lim e~ am = 1 . 

a—>0+ 
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Define y = (yi, ..., y m ) J '■= Ax o- By 0 < (3 < 1, we see that T C {1,..,, m} with \T C \ < /3m implies \T\ > 1. 
If uj^m > 0, it is trivial to see that 

^[0^, m ]^)<P{l2/ll + ' ' ' + \Vm\ > < 1; 

which is a contradiction to F(D| : 0a; ^ j ^) = 1. This proves wg im = lim Q ^. 0 + ^j3,m( a ) = 0. 

4.3. Proof of Theorem II.41 We are now ready to prove Theorem 11.41 


□ 


Proof of Theorem \1.4\ Observe that 0™ ax = lim Q ,_ s . 0 + #™ ax (a) and u;™ m = lim Q ,_ s . 0 + w™ m (a). Taking a —> 0 + 
in (14.11) of Lemma 14.11 we have 


(4.13) 


|(1 - P? min l) < o < min (l (In if, 1 


,4(1 -py~j ~ v 2 

This proves (11.211) . Taking a 0 + in (14.51) of Lemma 14.21 we have 
(4.14) /o - 2 


P 


max ( cl In ---, —/3 2 ) < < 2 In ■ 


9 l-P'2 ^ J 

This proves (11.221) . Similarly, taking a —>• 0 + in (14.91) of Corollary 14.31 we have 


(4.15) 


|(1 - P? min (4-2L, i) < if" < (1 - P) min (f (in -f)\ 1 ] . 


This proves (jl.23l) . We now prove (U.24p . Taking a —>• 0 + for the left-hand side of (14.101) in Corollary 14.41 
we proved the left-hand side of (ll.24p . We now prove the right-hand side of (ll.24p . Take IV G N such that 
N > j}. Let 7 := [(3m\/m. Then /3 — y- < (3 — A- < 7 < (3 for all m > N. Since (1 — x) In is a decreasing 
function on (0,1), we have (1 — (3 + j?) In -—— r > (1 — 7) In for all rri > N. Using the same notation 
as in Lemma 14.51 it follows from (|4.1ip in Corollary 14.41 that 


ujp(a) := sup u)p tTn (a) 
m£ N 


(4.16) 


< max (J2 (i - P + 4) In- 




Taking a —>• 0 + in the above inequality, we deduce from Lemma 14.51 that 

u /3 := lim up(a) < lim max (a), ..., u}p N -i(a), 2(1 - /3 + i)ln 

a—>-0+ ol —^0' l \ y 

=2( 1 -/3 + i) ln r — 

Since N > ^ can be arbitrarily large, combining with (|4.10l) . we proved 


1 ~ P + Tv 


——b V 2a\ 


(1 - /3) max |^c 5 in , ~P J < <up< 2(1 - 0) in 
This proves the left-hand side of (11,24|) . 


□ 


5. Proofs of Corollaries 

We now provide proofs to Corollaries 11.21 and 11.31 as well as the proofs of Corollaries 11.51 and 11.61 

Proof of Corollary 1 1.21 We assume that all the points pi,... ,pjy are distinct. For every T G T £)Q! , we have 
\T C \ < f3m for all 0 < /3 < By Theorem 13.51 for j / k, we have 


P 


II A T Pj - A T p k |p 


m\\pj -pk || 2 


- 1 


< e for all T G T e , a l > 1 - 3e -^ 2 /4-e 3 /6)m Vm G N. 
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Here we used the inequality 


1 — yfa e (1 — y / a)e 

0 < p < ---t- < V V ' 


32 In 


l — 


16 In- 


(1 -y/a)e 

provided that 0 < e < Since there are ((}) = — pairs {pj,Pk} with j / k, j, k = 1,..., N, using 

union bounds, we conclude that 


II A T Pj - A T p k f 


m\\pj -p k || 2 


- 1 


< e VT € T £)/3 , j / k, j,k = 1,... ,N \ > 1 


_ 3N(N - 1) ~_ a(e 2 /4 _ £ 3 /6)r 


> 0 , 


where we used the assumption of m > '‘afJv 4n inequality. This proves that (11.111) holds 


with probability at least 1 — 3JV (^ 1 ) e “L 2 / 4 e 3 /6)m > q 


□ 


Proof of Corollary \1JA We slightly modify the argument in [31 Lemma 5.1]. Let A C {1,..., n} with |A| = s. 
Set M a := {x G M n : x is supported inside A} and iS A := {x G M A : ||x|| = 1}. It is well known that there 
exists a subset Q\,e C S A such that |Qa,e| < (24 /e) s and S A C U(gQ Ae {x G : ||x — £|| < e/8}. By 
Theorem 13.51 with probability at least 1 — 3(24/e) s e _Q ( e2 / 16_e3//24 ) m , we have 


(5.1) 


V 7 ! - e /2|M| < —j=\\A t v\\ < y/Y/^eJ?., V T G T e/2a and v € Qa, £ - 
y/m ' 

We next consider the case where A satisfies (15.11) . Define A := sup{-^=||Agrx|| : x G S A ,T G T e / 2Q ,}. For 

every x G <S A , there exists v x G Q\,e such that ||x — u x || < e/8 and hence, 

7 = 1111 ^ 7 — H^T'i’xll H—t=||A(x — 'Lr)|| \/l + e/2 + A||a: — v x \\ < -\/l + e /2 + Ae/ 8 . 


/m y/m y/m 

By the definition of A, we must have A < yH + e/2 + Ae/8, which implies that 

\<^^<VTT~e 

for all 0 < e < 1. Therefore, for all x G M A and T G T e / 2 a , - 7 =||Arx|| < A||x|| < y/1 + e||x|| and 


\\A T x\\ > —=\\A t v x \\ - — \\A T {x - u x )|| > J1 - § - Af > J1 - f - l\/TTe > \/l - e, 


tm y/m y/m 

where the last inequality holds for all 0 < e < 1. Thus, with probability at least 1 —3(24/e) s e - “( £2 / 16 - e3 / 24 ) m , 
(5.2) (l-6)||x|| 2 <^||A T x|| 2 < (1 + e)||x|| 2 , V.Gl A ,TGT e/2 , a . 

Note that there are total (”) < ( en/s) s such subsets A. Therefore, (|5.2I) holds for every such subset A. By 
union bounds, (11.131) holds with probability at least 1 — 3(^|^) s e _ “F 2 / 16 - e3 / 2 4)m > q by our assumption 
sin < a(e 2 /16 — e 3 /24)m — In3. □ 


Proof of Corollary 11.51 The condition 0 < a < ^(1 — (3) 2 hp guarantees that 0 < 6 < oo, while the condition 
rri > guarantees 0 < u < oo (if m = j^a, then Oln | is understood as lim a ,^. 0 + ^Tn § = 0.) 

By the left-hand inequality in (14.9|) of Corollary 14.31 for any xq G M" 1 with ||®o|| = 1, we have P^MooL/S) > 
1 — e~ am . By (14.121) in Corollary 14.41 and (3 — — < 7 < /3, noting that (1 — x) In is a decreasing function 
on (0,1), we deduce that P(fi[o,cj],^) > 1 — e~ am . Consequently, we have 

P{0\\pj -Pk\\ 2 < ^\\A T Pj ~ A T p k \\ 2 < u\\pj -pfc|| 2 , VTC{l,...,m}, |T C | < /3m} > 1 - 2e -am , Vm G N 

for every j,k = 1,..., N. Since there are (^) = A(A ^ pairs {pj,Pj} with j / k, j, k = 1,..., N, we conclude 
that ([1.251) holds with probability at least 1 — N(N — l)e~ am > 0 for all m G N by rn > } In ■ □ 

Proof of Corollary ] 1.61 We use the same notation as in the proof of Corollary 11.31 Define T< f g := {T C 
{1,... ,m} : |T C | < /3m}. By Corollaries 14.31 and 14.41 with probability at least 1 — 2e~ am , 

(5.3) y/0(l - e/2)\\v\\ < -^=\\Atv\\ < + e/2), V T G T <p . 

Wm 
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We next consider the case where A satisfies (15.31) . Define A := sup{^=||H T x|| : x £ S A ,T £ T<p}. For 
every x £ <S A , there exists v x £ Qa,€ such that \\x — v x \\ < e/8 and hence, 

-^=\\A t x\\ < -^=\\A t v x \\ + —j=\\A(x - Ue)|| < yfu( 1 + e/2) + A||a: - v x \\ < y/u(l + e/2) + Ae/8. 

y/m y/m y/m 

By the definition of A, we must have A < y/u:( 1 + e/2) + Ae/8, from which we have A < y/u>( 1 + e/2)/(l — 
e/8) < y/oo( 1 + e) for all 0 < e < 1. Therefore, for all x £ M A and T £ T< u , -A=||^4 T x|| < A||x|| < 
y/u)(l + e)||x|| and 

~^=\\A t x || > -^=\\A t v x \\ - ]=\\A t {x - Ur)II > Vdy/l - e/2 - Ae/8 

y/m y/m y/m 

> Ve (yi - e/2 - v437^\/rTe(e/8)) > \/0 (V 1 ~ e/2 - vT+e(e/8)) > 

where we used the fact that u/9 >1 by 0<6*<w. Thus, with probability at least 1 — 2(24/e) s e - “ m , 

(5.4) 0(l-e)||x|| 2 < ip T ®|| 2 <a;( 1 + e)||x|| 2 , Vx€M A ,TeT<^. 

Note that there are total (™) < ( en/s) s such subsets A. Therefore, (|5.4|) holds for every such subset A. 
Hence, (11.131) holds with probability at least 1 — 2(^^) s e~ am > 0 by sln^p < am — In 2. □ 
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